1. Technical Field
The technology presented herein relates to an image processing system, a storage medium storing an image processing program, an image processing apparatus and an image processing method. More specifically, the present technology relates to an image processing system, a storage medium storing an image processing program, an image processing apparatus and an image processing method which display a composite image obtained by combining a photographed image and a CG image.
2. Description of the Related Art
As an example of this kind of a conventional image processing apparatus, one disclosed in a Japanese Patent Application Laid-Open No. 2000-322602 is known. In the related art, the image processing apparatus images a card attached with a two-dimensional bar code with a CCD camera, searches the two-dimensional bar code from this imaged image, detects a position of the two-dimensional bar code in the imaged image, identifies a pattern of the two-dimensional bar code, and then displays a three-dimensional image according to this pattern so as to be superposed at the position of the two-dimensional bar code within the imaged image.
In the aforementioned related art, the two-dimensional bar code is printed on a card, but it is conceivable that this is displayed on a monitor of another image processing apparatus, such as a monitor of a PC, for example. In this case, by changing the two-dimensional bar code on the monitor via the PC, it becomes possible to make the image processing apparatus combine various three-dimensional images. However, this is merely a one-way display control from the PC to the image processing apparatus, and has a limit to enhancement of savor of the game.
Therefore, it is a primary feature of the present technology to provide a novel image processing system, a novel storage medium storing an image processing program, a novel image processing apparatus and a novel image processing method.
Another feature of the present technology is to provide a image processing system, a storage medium storing an image processing program, an image processing method, a first image processing apparatus, and a second image processing apparatus which can bring the first image processing apparatus and the second image processing apparatus into association with each other to thereby operatively connect a first object and a second object between a first displayer and an imaged image of a second displayer.
The present technology employs following features in order to solve the above-described problems.
The present technology adopts the following construction in order to solve the above-mentioned problems.
A first aspect is an image processing system including a first image processing apparatus utilizing a first displayer and a second image processing apparatus utilizing an imager and a second displayer capable of viewing a real space on a screen, wherein the first image processing apparatus comprising: a first display processor which displays a predetermined marker image on the first displayer; and a first object display controller which performs on the first displayer a display control of at least a part of first object image being a predetermined CG object; and the second image processing apparatus comprising: an imaging processor which performs imaging by the imager; and a second object display controller which performs a composition control of at least a part of second object image being a predetermined CG object on a real space capable of being viewed on the second displayer at a position with reference to the marker image within the imaged image by recognizing the marker image within the imaged image.
In the first aspect, a first image processing apparatus displays a predetermined marker image on the first displayer to thereby perform a display control of at least a part of first object image being a predetermined CG object, and a second image processing apparatus performs imaging by the imager, and recognizes a marker image within the imaged image to thereby perform a display control of at least a part of second object image being a predetermined CG object at a position with reference to the marker image within the imaged image on the real space capable of being viewed on the second displayer.
According to the first aspect, the first image processing apparatus and the second image processing apparatus are brought into association with each other via the marker image to thereby operatively connect the first object and the second object between the first displayer and the imaged image of the second displayer.
Here, the first object and the second object may be a common object (that is, the common object becomes the first object on the first displayer, and becomes the second object on the second displayer), and may be a part of the common object (head and arms, for example) and other parts (body, for example). Thus, the first object and the second object may preferably be objects that are brought into association with each other, but may be objects independent of each other.
For example, in a case that the first object and the second object are the common object (a part thereof and another part thereof), the object can look as if it moves between the first displayer and the second displayer (perform a display control). In one embodiment, the object pops out of the first displayer to the imaged image of the second displayer, or returns to the first displayer therefrom. In another modified example, the objects appear on the first displayer and the imaged image of the second displayer at the same time and disappear therefrom at the same time.
Additionally, it is preferable that the marker image is always displayed, but displayed as required. The marker image is always displayed in one embodiment, but it is displayed as required in a modified example.
Furthermore, the composition is a display in a superimposed manner, but the imaged image itself may be changed.
A second aspect is according to the first aspect, wherein the first image processor and the second image processor are able to communicate with each other, the first object display controller performs a display control of the first object image by being operatively connected through the communication, and the second object display controller performs a composition control of the second object image by being operatively connected through the communication.
According to the second aspect, the first image processing apparatus and the second image processing apparatus are brought into association with each other via the marker image and the communications, capable of enhancing the continuity between the first object and the second object.
A third aspect is according to the second aspect, wherein the second object display controller combines at least a part of the second object image with the imaged image when the marker image is recognized within the imaged image, and the first object display controller performs a control on the first displayer such that the first object image disappears when the marker image is recognized within the imaged image in the second object display controller.
According to the third aspect, in accordance with the first object image disappearing from the first displayer, the second object image can be made to appear within the imaged image of the second displayer.
A fourth aspect is according to the second aspect, wherein the second object display controller combines at least a part of the second object image with the imaged image when the marker image is recognized within the imaged image, and the first object display controller performs a control such that the first object image is displayed on the first displayer when the marker image is recognized within the imaged image in the second object display controller.
According to the fourth aspect, in accordance with the second object image appearing within the imaged image on the second displayer, the first object image can also be made to appear in the first displayer.
A fifth aspect is according to the first aspect, wherein the marker image includes identification information; and the first object image and the second object image are images corresponding to the identification information included in the marker image.
According to the fifth aspect, through the identification information included in the marker image, various first object images and second object images can be displayed.
A sixth aspect is according to the first aspect, wherein the first display processor displays the plurality of marker images on the first displayer.
According to the sixth aspect, by displaying the plurality of marker images, it is possible expand a recognizable range.
A seventh aspect is according to the sixth aspect, wherein the first display processor displays four marker images at four corners of the first displayer.
According to the seventh aspect, it is possible expand the recognizable range with the visibility of the first displayer maintained as high as possible.
An eighth aspect is according to the sixth aspect, wherein the first object display controller performs a control such that the first object image is displayed at a predetermined position surrounded by the plurality of marker images, and by recognizing at least one of the plurality of marker images within the imaged image, the second object display controller performs a control of a composition of the second object image on a position surrounded by the plurality of marker images recognized within the imaged image.
According to the eighth aspect, it is possible to display the object image at a predetermined position surrounded by the markers on each of the first displayer and the second displayer.
A ninth aspect is according to the first aspect, wherein the second object display controller which performs a composition control of the second object at a position and an orientation corresponding to a position and an orientation of the marker image within the imaged image by performing an AR recognition on the marker image within the imaged image.
According to the ninth aspect, it is possible to accurately perform a composition control of the second object through the AR recognition.
A tenth aspect is according to the ninth aspect, wherein the second object display controller includes: a position and orientation calculator which calculates a correlative relation of a position and an orientation between the marker images on the first displayer and the imager by recognizing the marker image within the imaged image; a virtual camera setter which arranges the second object in the virtual space and decides a position and an orientation of the virtual camera such that a correlative relation of a position and an orientation between the second object and the virtual camera match the position and the orientation that are calculated by the position and orientation calculator; and a virtual space imager which images the virtual space including the second object by the virtual camera, wherein a composition control is performed between the imaged image and the virtual space imaged by the virtual space imager.
According to the tenth aspect, a correlative relation of a position and a orientation between the marker image on the first displayer and the imager is calculated to thereby arrange the second object in the virtual space, and the position and orientation of the virtual camera are decided a correlative relation of the position and the orientation between the second object and the virtual camera match the position and orientation calculated by the position and attitude calculator, capable of combining the second object with the imaged image with high accuracy.
An eleventh aspect is according to the first aspect, wherein the second image processing apparatus further comprises a second display processor which displays the imaged image imaged by the imaging processor on the second displayer, and the second object display controller performs a composition control of the second object on the imaged image displayed by the second displayer.
According to the eleventh aspect, it is possible to combine the second object with the imaged image.
A twelfth aspect is according to the first aspect, wherein the second image processing apparatus further comprises: a first signal transmitter which transmits a first signal to the first image processing apparatus on the basis of a recognition result of the marker image, and the first image processing apparatus further comprising: a first signal receiver which receives the first signal transmitted by the first signal transmitter, wherein the first object display controller controls a display of the first object on the basis of the first signal received by the first signal receiver.
According to the twelfth aspect, the first image processing apparatus displays a predetermined marker image on a first displayer to thereby make the second image processing apparatus to perform a display control of the second object on the imaged image of the second displayer while the second image processing apparatus transmits a first signal when the display control is performed on the basis of the markers to thereby make the first image processing apparatus perform a display control of the first object on the first displayer. Thus, the first image processing apparatus and the second image processing apparatus are associated with each other through the marker image and the first signal to thereby operatively connect the first object and the second object between the first displayer and the imaged image of the second displayer.
Here, the first signal is preferably a determination result signal indicating a determination result (YES) that the marker image is included, and it is repeatedly transmitted, but it may be a timing signal indicating that the determination result changes from “NO” to “YES”, and it may be transmitted by one.
A thirteenth aspect is according to the twelfth aspect, wherein the first image processing apparatus further comprises: a second signal transmitter which transmits a second signal to the second image processing apparatus in a case that the first signal is received by the first signal receiver, and the second image processing apparatus further comprises: a second signal receiver which receives the second signal, wherein the second object display controller performs a display control on the basis of the second signal received by the second signal receiver.
In the thirteenth aspect, the second image processing apparatus transmits the second signal to the first image processing apparatus when the first signal is received, and the first image processing apparatus performs a display control on the basis of the second signal.
According to the thirteenth aspect, the first image processing apparatus and the second image processing apparatus are associated with each other through the second signal in addition to the marker image and the first signal, capable of enhancing the continuity between the first object and the second objet.
Here, the second signal is preferably a control signal indicating the content and a timing of the display control, but it may merely be a timing signal indicating a timing of the display control, and may be an acknowledgment signal (ACK, for example) confirming reception of the first signal. Furthermore, the second signal may preferably be transmitted repeatedly, but may be transmitted by one.
A fourteenth aspect is according to the thirteenth aspect, wherein the second signal transmitter transmits the second signal to the second image processing apparatus after a lapse of a first predetermined time since the first signal is received by the first signal receiver.
According to the fourteenth aspect, the first controlling device transmits the second signal on the basis of the elapsed time from the reception of the first signal, and therefore, it is possible to perform an active control in relation to the composition of the second object.
Alternatively, the transmission of the second signal by the first image processing apparatus may passively be performed on the basis of a command from the first controlling device.
A fifteenth aspect is according to the thirteenth aspect, wherein the first object display controller performs a display control after the second signal is transmitted by the second signal transmitter.
According to the fifteenth aspect, the first image processing apparatus performs the display control of the first object after transmitting the second signal. In response to the second signal, the second image processing apparatus performs the display control of the second object. Thus, it is possible to synchronize the display control by the first image processing apparatus itself and the display control by the second image processing apparatus. For example, it becomes easy to match a starting timing of the display control between the first image processing apparatus and the second image processing apparatus.
A sixteenth aspect is according to the fourteenth aspect, wherein the first image processing apparatus further comprises a third signal transmitter which transmits a third signal to the second image processing apparatus after a lapse of a second predetermined time since the first object display controller performs a display control, the second image processing apparatus further comprises a third signal receiver which receives the third signal, and the second object display controller erases the second object from the imaged image after the third signal is received by the third signal receiver.
According to the sixteenth aspect, the first controlling device transmits the third signal on the basis of the elapsed time since the display control of the first object is performed, and therefore, it is possible to actively perform a control on the composition and moreover the erasure of the second object.
Alternatively, the transmission of the third signal by the first image processing apparatus may passively be performed on the basis of a command from the first controlling device.
Here, the third signal is preferably a control signal indicating the content and a timing of the display control, but it may merely be a timing signal indicating a timing of the display control. Furthermore, the third signal may preferably be transmitted repeatedly, but may be transmitted by one.
A seventeenth aspect is according to the sixteenth aspect, wherein the first object display controller returns to a state before the display control is performed after the third signal is transmitted by the third signal transmitter.
According to the seventeenth aspect, when the second object is erased on the second displayer, the imaged image on the first displayer can be returned to the original state, that is, the state before the first object is combined.
An eighteenth aspect is according to the twelfth aspect, wherein the first display processor displays at least a part of the first object together with the predetermined identification information, and the first object display controller erases at least a part of the first object on the basis of the first signal.
According to the eighteenth aspect, it becomes possible to make an expression as if the object moves between the first displayer and the second displayer.
A nineteenth aspect is according to the twelfth aspect, wherein the first object display controller displays at least a part of the first object on the basis of the first signal.
According to the nineteenth aspect, it becomes possible to make an expression as if the objects appear and/or disappear between the first displayer and the second displayer at the same time.
A twentieth aspect is according to the first aspect, wherein the first object display controller includes a first object size changer which changes a size of the first object on the basis of a size of the marker image, and the second object display controller includes a second object size changer which changes a size of the second object on the basis of the size of the marker image.
According to the twentieth aspect, it becomes possible to control the size of the object through the size of the marker.
A twenty-first aspect is according to the first aspect, wherein the first object display controller includes a first object direction changer which changes a display direction of the first object on the basis of a shape of the marker image, and the second object display controller includes a second object direction changer which changes a display direction of the second object on the basis of the shape of the marker image.
According to the twenty-first aspect, it becomes possible to control the direction of the object through the shape of the marker.
A twenty-second aspect is according to the twelfth aspect, wherein the first signal includes coordinate information, and the first object display controller performs a display control of at least a part of the first object on the basis of the coordinate information included in the first signal.
According to the twenty-second aspect, the second image processing apparatus transmits the first signal inclusive of the coordinate information, and whereby, the first image processing apparatus can make a display in association with the second image processing apparatus.
A twenty-third aspect is according to the eleventh aspect, wherein the second display processor includes a frame displayer which displays a frame the same in shape as the marker image on the second displayer, and the second object display controller performs recognition in a state that the marker image is displayed along the frame displayed by the frame displayer.
According to the twenty-third aspect, a frame the same in shape as the marker image is displayed on the second displayer, and a recognition is performed with the marker images displayed along the frame, and whereby, it saves the processing of a direction detection based on the position of the marker image and further a coordinate transformation, capable of reducing the processing load.
A twenty-fourth aspect is an image processing program performing image processing between a first image processing apparatus utilizing a first displayer and a second image processing apparatus utilizing an imager and a second displayer capable of viewing a real space on a screen, wherein the image processing program causes a computer to function as: a first display processor which displays a predetermined marker image on the first displayer; and a first object display controller which performs on the first displayer a display control of at least a part of first object image being a predetermined CG object, the image processing program causes the second image processing apparatus to function as: an imaging processor which performs imaging by the imager; and a second object display controller which performs a composition control of at least a part of second object image being a predetermined CG object on a real space capable of being viewed on the second displayer at a position with reference to the marker image within the imaged image by recognizing the marker image within the imaged image.
A twenty-fifth aspect is a first image processing apparatus being brought into association with a second image processing apparatus that utilizes an imager and a second displayer capable of viewing a real space on a screen by utilizing a first displayer, comprising: a first display processor which displays a predetermined marker image on the first displayer; and a first object display controller which performs a display control of at least a part of first object image being a predetermined CG object; wherein the second image processing apparatus comprises an imaging processor which performs imaging by the imager; and a second object display controller which performs a composition control of at least a part of second object image being a predetermined CG object on a real space capable of being viewed on the second displayer at a position with reference to the marker image within the imaged image by recognizing the marker image within the imaged image.
A twenty-sixth aspect is a second image processing apparatus utilizing an imager and a second displayer capable of viewing a real space on a screen in associating with a first image processing apparatus utilizing a first displayer, wherein the first image processing apparatus comprises: a first display processor which displays a predetermined marker image on the first displayer; and a first object display controller which performs on the first displayer a display control of at least a part of first object image being a predetermined CG object, comprising: an imaging processor which performs imaging by the imager; and a second object display controller which performs a composition control of at least a part of second object image being a predetermined CG object on a real space capable of being viewed on the second displayer at a position with reference to the marker image within the imaged image by recognizing the marker image within the imaged image.
A twenty-seventh aspect is an image processing method performed by a first image processing apparatus utilizing a first displayer and a second image processing apparatus utilizing an imager and a second displayer capable of viewing a real space on a screen, including followings steps to be executed by a computer of the first image processing apparatus: a first display processing step for displaying a predetermined marker image on the first displayer; and a first object display controlling step for performing on the first displayer a display control of at least a part of first object image being a predetermined CG object, and including following steps to be executed by a computer of the second image processing apparatus of: an imaging processing step for performing imaging by the imager; and a second object display controlling step for performing a composition control of at least a part of second object image being a predetermined CG object on a real space capable of being viewed on the second displayer at a position with reference to the marker image within the imaged image by recognizing the marker image within the imaged image.
In the twenty-fourth to twenty-seventh aspect as well, similar to the first aspect, the first image processing apparatus and the second image processing apparatus are brought into association via the marker image to thereby operatively connect the first object and the second object between the first displayer and the imaged image of the second displayer.
According to the technology presented herein, it is possible to implement an image processing system, an image processing program and an image processing method, and a first image processing apparatus and a second image processing apparatus therefor capable of operatively connecting the first object and the second object between the first displayer and the imaged image of the second displayer by bringing the first image processing apparatus and the second image processing apparatus into association with each other.
The above described features, aspects and advantages of the present technology will become more apparent from the following detailed description of the present technology when taken in conjunction with the accompanying drawings.